Learning from the hindsight plan - Episodic MPC improvement

نویسندگان

  • Aviv Tamar
  • Garrett Thomas
  • Tianhao Zhang
  • Sergey Levine
  • Pieter Abbeel
چکیده

Model predictive control (MPC) is a popular control method that has proved effective for robotics, among other fields. MPC performs re-planning at every time step. Replanning is done with a limited horizon per computational and real-time constraints and often also for robustness to potential model errors. However, the limited horizon leads to suboptimal performance. In this work, we consider the iterative learning setting, where the same task can be repeated several times, and propose a policy improvement scheme for MPC. The main idea is that between executions we can, offline, run MPC with a longer horizon, resulting in a hindsight plan. To bring the next real-world execution closer to the hindsight plan, our approach learns to re-shape the original cost function with the goal of satisfying the following property: short horizon planning (as realistic during real executions) with respect to the shaped cost should result in mimicking the hindsight plan. This effectively consolidates long-term reasoning into the shorthorizon planning. We empirically evaluate our approach in contact-rich manipulation tasks both in simulated and real environments, such as peg insertion by a real PR2 robot.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Can Causal Sense-Making Benefit Foresight, Rather than Biasing Hindsight?

Upon reading headlines like “Traffic Fatalities Increased/Decreased Last Year,” people often overestimate how well they would have anticipated changes. This hindsight bias has been linked to causal sensemaking that minimizes one’s feeling of surprise after learning an outcome. In this paper, we consider whether the sensemaking process, which contributes to bias in hindsight, could be recruited ...

متن کامل

Explaining individual differences in cognitive processes underlying hindsight bias.

After learning an event's outcome, people's recollection of their former prediction of that event typically shifts toward the actual outcome. Erdfelder and Buchner (Journal of Experimental Psychology: Learning, Memory, and Cognition, 24, 387-414, 1998) developed a multinomial processing tree (MPT) model to identify the underlying processes contributing to this hindsight bias (HB) phenomenon. Mo...

متن کامل

Hindsight bias from 3 to 95 years of age.

Upon learning the outcome to a problem, people tend to believe that they knew it all along (hindsight bias). Here, we report the first study to trace the development of hindsight bias across the life span. One hundred ninety-four participants aged 3 to 95 years completed 3 tasks designed to measure visual and verbal hindsight bias. All age groups demonstrated hindsight bias on all 3 tasks; howe...

متن کامل

SACS, QEP, and Hindsight

As institutions in the Southeastern section prepare for reaffirmation of accreditation by the Southern Association of Colleges and Schools (SACS), fundamental changes have been implemented that have a significant impact on the institution and academic programs. One of the major additions is SACS core requirement 2.12 that requires an institution to prepare a Quality Enhancement Plan (QEP). The ...

متن کامل

Multiclass Classification Based on Meta Probability Codes

This paper proposes a new approach to improve multiclass classi ̄cation performance by employing Stacked Generalization structure and One-Against-One decomposition strategy. The proposed approach encodes the outputs of all pairwise classi ̄ers by implicitly embedding twoclass discriminative information in a probabilistic manner. The encoded outputs, called Meta Probability Codes (MPCs), are inter...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2017